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ABSTRACT 

Adaptive aiding is an idea that offers potential for 
improvement over many current approaches to aiding in 
human-computer systems. The expected return of tailoring the 
system to fit the user could be in the form of improved system 
performance and/or increased user satisfaction. Although the 
utility of the concept has been demonstrated in limited ways in a 
variety of contexts, there has been no substantial research 
effort devoted to addressing the many issues relevant to adaptive 
aiding. These include such issues as the manner in which 
information is shared between human and computer, the appropriate 
division of labor between them, and the level of autonomy of the 
aid . 


In order to investigate these and other issues relevant to 
human-computer interaction, a simulated visual search task has 
been developed. Subjects are required to identify targets in a 
moving display while performing a compensatory sub-critical 
tracking task. It is also possible for the computer to identify 
targets. By manipulating characteristics of the situation such 
as imposed task-related workload and effort required to 
communicate with the computer, it is possible to create 
conditions in which interaction with the computer would be more 
or less desirable. The results of preliminary research using 
this experimental scenario' are presented, and future directions 
for this research effort are discussed. 


INTRODUCTION 

The idea of providing the human operator with some form of 
computer assistance is not new. Computers have been used for 
years in a variety of applications. Often the complexity of 
modern systems and the potentially high costs of system failure 
have been invoked as justification for computerizing portions of 
the operator's job. 

The decision as to which tasks will be performed by computer 
has all too often been based upon which tasks could be automated. 
In situations where total automation was not feasible, the task 
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allocation decision has been based upon relative abilities of 
human and computer. For example, humans would be given tasks 
requiring "flexibility" and computers would perform tasks 
requiring "consistency". A number of lists of human vs, 
computer abilities are available for this purpose (e.g., 
Licklider, I960). 

For several reasons, this "traditional" approach to computer 
aiding may be less than satisfactory. For example, thanks to 
progress in artificial intelligence, the distinction between 
human and computer abilities is much less clear. Thus, the human 
and computer may be viewed as partners, with abilities which 
partially overlap. As a result, it may be inappropriate to 
allocate tasks based solely on computer abilities. 

Another factor which should be considered is individual 
differences. Aptitudes and abilities, cognitive styles, and 
attitudes have been cited as affecting human behavior in a number 
of situations. Lists of human abilities are characteristic of a 
prototypical human and do not reflect these differences. 

Human performance varies not only across individuals but 
also within individuals over time. People become fatigued. They 
have a limited capacity to perform. The "mix" of required tasks 
may impose an inordinate amount of workload, and performance may 
degrade as a result. 

Finally, the quality of the computer’s performance may 
depend upon conditions. For example, suppose the computer must 
have certain state information in order to make decisions. If 
the quality of that information is degraded, performance of the 
computer will be affected. 

In light of these shortcomings, it seems desirable to make 
computer aids adaptive. An adaptive aid could step in when 
needed and provide assistance in a form appropriate to the 
situation. . In situations where no assistance was needed, the aid 
could remain inactive. In principle, it seems that such an 
approach to aiding could improve overall system performance 
substantially. 


RELEVANT ISSUES 

The concept of adaptive aiding is also not new (Chu 6 Rouse, 
1979; Rouse, 1975, 1981). However, it has not been implemented 
in any real-world applications, probably because the manner in 
which this should be done is not at all straightforward. A 
number of issues must be considered before progress can be made 
(Rouse & Rouse, 1983). For example, what should the focus of 
adaptation be? Should the aid be adapted to group 
characteristics, or to individuals? Should adaptation be done 
once, or dynamically over time? 
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Another issue is the method of adaptation. At least three 
approaches are imaginable. Tasks may be allocated, with either 
the human or computer in control of task performance. 
Alternatively, tasks may be partitioned between the two partners, 
with each performing task components. Finally, one partner may 
assist the other by performing a transformation of a task (e.g., 
the computer could filter noise from a visual display). 

If human and computer are to be partners, then there must be 
some means for the two to communicate. But what should be the 
nature of communication? If communication is explicit, there is 
less uncertainty as to what is being communicated, but the human 
must invest resources in receiving and transmitting information. 
This resource demand may be less if communication is implicit, 
but there may be less certainty as to what is communicated. 
There may also be a need for the human to invest resources into 
determining what the computer is doing. 

When system control is shared by human and computer, which 
partner should be in charge ? Suppose tasks are to be allocated 
dynamically. Which partner — human or computer — should make the 
decision as to task allocation? As with the nature of 
communication, the resources required to make decisions and 
inform the partner must be considered. 

Finally, if it appears that it would be advantageous to have 
the computer make decisions such as task allocation, what is the 
basis for decision making? It will be necessary to imbed models 
Tn The computer T s knowledge base if such decisions are to be 
possible. These models must incorporate characteristics of the 
task situation, the human's task performance, and the computer's 
performance in order to be effective. Although the results of 
research in human problem solving and information processing 
provide a partial data base to support such models, many 
parameters must be obtained via specific research in 
human-computer interaction. The goal of the work reported here 
is to investigate these and other relevant issues. 

EXPERIMENTAL APPROACH 

In explaining the approach adopted in this research effort, 
it helps to consider a hypothetical situation. Suppose a variety 
of tasks must be performed for overall system operation to be 
successful. Human performance of these tasks on an individual 
basis is acceptable, but the degree to which tasks may be 
time— shared successfully depends upon the level of difficulty and 
combination of concurrent tasks. Further, suppose a computer is 
available which may perform a subset of these tasks. The 
computer's task performance may or may not be as good as the 
human's best performance, but may be preferable if the human's 
performance degrades. 

An attempt was made to create this situation experimentally. 


457 



In designing the experimental scenario, one goal was to maintain 
a semblance of realism, rather than create. an "artificial" 
laboratory task. However, the characteristics of the task 
environment were determined analytically, and little attempt was 
made to provide a high-fidelity simulation of an actual task. .•» 

A target recognition task was created as one of the tasks in 
the scenario because of differences in human and. computer 
abilities in this area. Humans readily impart meaning into what 
is seen, and are excellent at "perceptual organization". 
Computers, on the other hand, have a great deal, of difficulty 
analyzing scenes, but excel at figure rotation and template 
matcning. Thus, humans should be better at identifying features 
in a meaningful scene, whereas computers should be better if the 
scene is a relatively homogenous field of objects. 

Description of Experimental Tasks 

The target recognition task employs a color graphic terrain 
display, as illustrated in Figure 1. The terrain display depicts 
an intracoastal waterway with varying proportions of water. 
Water areas are colored blue. Also included in the terrain are 
green trees, tan ground, black buildings, white roads and parking 
lots, and cars and boats of assorted colors. To simulate flight 
over the terrain the display pans down the CRT. Subjects are 
given the goal of identifying or spotting boats of a certain type 
which are in use in the waterway. 

Targets may be identified only when they are in the region 
defined by the heavy black horizontal lines. When the subject is 
identifying targets, identification is accomplished by using a 
mouse to position the cross-hair cursor on top of the target and 
then pressing a button on the mouse. When the button is pressed 
a •* + " appears on the screen to acknowledge the action. Hits and 
false alarms are tallied in the upper left corner of the screen. 
(See Figure 1 . ) 

It is also possible for the computer to perform the spotting 
task. If the human is in control of the allocation decision, the 
aid may be activated by positioning the cursor on top of the word 
"AID" (to the left of the terrain display in Figure 1), and 
pressing the button on the mouse. The cursor then disappears, 
and the aid identifies targets until the human resumes control by 
again pressing the button on the mouse. 

The relative performance of human and computer may be 
expected to vary over time. In light of the human's perceptual 
abilities, this task should be easier for the human when * he 
proportion of water in the picture is low (such as. when flying 
over a narrow channel). This is because the human is able to 
organize the scene and automatically exclude a large portion 
(i.e., the land areas) from consideration. 
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The computer, on the other hand, is deficient in these 
organizational abilities, and scans the whole scene, identifying 
boats with a "template matching" approach.* As a result, the 
computer does not always differentiate land from water, and its 
false alarm rate increases with the proportion of land in the 
display. Thus, the human may be expected to excel when the 
proportion of water is low, and there is greater potential for 
the aid to excel when the proportion of water is high. 

Target identification is not the only task which must be 
performed. In addition to looking for boats, the human must also 
perform a subcritical tracking task. The tracking display is 
shown in the upper left corner of Figure 1 . 

The tracking display contains a green region flanked by 
yellow and red regions. The horizontal black line to the right 
of these regions moves up and down, and the arrow within the 
green region indicates the direction of the control input. The 
degree of instability of the controlled element is determined by 
a difficulty parameter which is entered by the experimenter at 
the beginning of a run and remains constant throughout the run. 
The human's goal is to keep the black line within the green 
region by using bang-bang control via the space bar on the 
terminal keyboard. When performing both tasks, the subject 
identifies targets with the right hand and tracks with the left. 

The primary reason for incorporating the tracking task into 
the scenario is to create conditions in which assistance from the 
computer is required in order to maintain satisfactory 
performance. If target identification were the only task 
required, it is conceivable that a subject could maintain 
acceptable performance over a wide range of difficulty. However, 
performance should be more sensitive to difficulty manipulations 
(i.e., changes in terrain composition) if tracking is also 
required. The difficulty parameter of the tracking task may be 
varied to insure that such is the case, and the option of 
"shedding" the tracking task in favor of the target 
identification task is eliminated by disabling mouse inputs 
whenever the tracking indicator is in a red region. 

With respect to the adaptive aiding concept, it is possible 
to specify qualitatively when the computer should be used in this 
environment. First, the aid should be used if its potential 
target identification performance exceeds that of the human. It 
is expected that this occurrence is most likely when tracking is 
non- trivial and the terrain is mostly water. Second, the aid 
should be used to look for boats if the human's tracking 


*In reality the computer "knows" the identity and location 
of every object in the display and makes responses on a 
probabilistic basis. The template matching explanation is 
provided to subjects. 
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performance degrades to an unacceptable level. Excluding the 
case in which acceptable tracking is impossible due to the level 
of tracking difficulty, it is anticipated that this, occurrence 
would also be related to the amount of water in the display. 

AN EXPERIMENT 

A pilot study was conducted to evaluate the accuracy of some 
of these ideas by assessing the effects of task parameters on 
subject's performance. Since one of the purposes of this 
experiment was to identify conditions in which the need for 
computer assistance would be likely, no aid was available to 
subj ects . 

Two subjects served in three sessions each. The first 
session served as training and consisted of one 5-minute run at 
each of four levels of tracking difficulty. In the second and 
third sessions, the easiest tracking condition was excluded and 
only three levels of tracking difficulty were used. Thus, there 
were two independent variables in the pilot study: tracking 
difficulty and terrain composition. Dependent measures included 
rms tracking error, spotting accuracy (i.e., percent identified) 
and spotting latency (i.e., average time to identify a target 
once it entered the spotting window). 

The results of this study are presented graphically in 
Figures 2-4. Time is represented on the abscissa of each graph, 
as the values shown represent the sequence of terrain types 
encountered by subjects over the course of a run. One interval 
on the abscissa corresponds to approximately 20 seconds of real 
time. To facilitate interpretation of these figures, terrain is 
also identified as either predominantly land or predominantly 
water. The break or dashed line in the middle of each graph 
reflects missing data. Due to hardware constraints, targets in 
these areas are not accessible to subjects, and there is a 1-2 
second interval of "dead time" in the middle of each run. 

Figure 2 depicts rms tracking error for three levels of 
tracking difficulty, averaged across both subjects. Two 
characteristics of Figure 2 are noteworthy. First, rms tracking 
error increased with increases in the difficulty parameter of the 
tracking task. Second, rms tracking error increased with the 
amount of water in the display. This effect seems to have been 
stronger when tracking was relatively easy, but is noticeable at 
each of the levels of tracking difficulty employed in this study. 

From Figures 3 and 4» it may be ascertained that performance 
on the target identification task was also affected by changes in 
the terrain composition. Increases in the proportion. of water in 
the display were accompanied by decreases in spotting accuracy 
(although small) and increases in spotting latency. Unlike rms 
tracking error, there was no noticeable effect of tracking 
difficulty manipulations upon target identification; as a 
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result, the plots in Figures 3 and 4 represent performance 
averaged across three levels of tracking difficulty. 

If the three dependent measures are compared to each other, 
some clear relationships emerge. First, there is an obvious 
negative relationship between spotting accuracy and spotting 
latency. Product-moment correlations at different levels of 
tracking difficulty ranged from -.61 to -.70. Of course, these 
results were obtained with only two subjects, so generalizations 
should be made with caution; however, if further experiments 
continue to reveal this relationship, this may have implications 
for online adaptation. 

Although spotting accuracy is the stated performance 
criterion, its utility as an online measure is limited due to two 
factors. First, observed decrements in spotting accuracy were 
quite small, usually no more than 2-3 missed targets. Second, it 
seems desirable to be able to offer assistance before a target is 
missed, rather than stepping in too late to cTo any good. 
Spotting latency is easily assessed online; if the relationship 
of latency to accuracy proves to be sufficiently strong, the 
latency measure may be useful as a basis fop online computer 
adaptation. 

It may also be noted that rms tracking error is related to 
both spotting accuracy and spotting latency. Since it is an 
easily calculated, continuous measure, rms tracking error may 
also be useful as a basis for decision making. However, the 
results from this pilot study indicate that rms tracking error 
may not be as useful for this purpose as spotting latency, 
because its response to task changes considerably lags the 
response of spotting latency to these changes. (A comparison of 
Figures 2 and 4 reveals a difference of almost 20 seconds in the 
most difficult tracking condition.) 

PLANS FOR FUTURE RESEARCH 

Current plans are to conduct a full-scale experiment this 
summer. Independent variables will be the same as those reported 
here: terrain composition and tracking difficulty. 
Additionally, an initial attempt will be made to have the 
computer make the decision as to allocation of the target 
identification task. Undoubtedly the decision algorithm will be 
rather simplistic; however, this should provide insights 
necessary for more effective decision aiding in the future. 

At present, it is possible to imagine several alternative 
approaches to allocation which might be appropriate. For 
example, in addition to unilateral decision making by human or 
computer, a "hybrid" approach could prove to be useful. In this 
case, the computer could monitor the human's performance and 
assume control of the target identification task when his 
performance on either task began to degrade. The human could 
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then resume control of target identification when he felt able to 
do so. 


If online adaptation is to be effective, it will be 
necessary to identify appropriate measures to serve as the bases 
for decision aiding, and to develop adequate models of how 
important variables interact. Effort will be devoted to 
achieving both of these goals. Identification of measures will 
be approached in a manner similar to that described here, by 
obtaining multiple performance measures and noting relationships 
between intermediate behavior and ultimate performance. A 
preliminary conceptual model of human-computer interaction has 
been developed (Morris, Rouse, & Ward, 1984, in preparation), and 
will be evaluated as research results become available. An 
’’armchair” analysis of the problem indicates that such a model 
should include not only aspects of the task situation but also 
should take into account such factors as the human’s perception 
of his own and the computer's performance, and human information 
processing resource limitations. 

Also of interest are a number of issues relevant to problems 
which may arise when the computer aid degrades in some way. For 
example, under what conditions will the human realize that the 
aid has degraded, and will it be possible for the human to cope 
with the loss of the aid? Investigation of these and other 
questions may entail consideration of knowledge requirements and 
the human's "mental models" of the aid and situation. 
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Figure 1. Task display. 



RMS TRACKING ERROR 



Figure 2. RMS tracking error. 
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SPOTTING ACCURACY ( % IDENTIFIED) 




SPOTTING LATENCY (AVG, TIME TO IDENTIFY, SEC.) 



